Gemma/[Gemma_3]Using_with_Transformersjs.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BVp8vazXYOz-"
      },
      "source": [
        "##### Copyright 2025 Google LLC."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "cellView": "form",
        "id": "DBaXaQ_PYT4p"
      },
      "outputs": [],
      "source": [
        "# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XSkl5h3dZMo9"
      },
      "source": [
        "# Gemma 3 - Run with Transformers.js\n",
        "\n",
        "Author: Sitam Meur\n",
        "\n",
        "*   GitHub: [github.com/sitamgithub-MSIT](https://github.com/sitamgithub-MSIT/)\n",
        "*   X: [@sitammeur](https://x.com/sitammeur)\n",
        "\n",
        "Description: This notebook demonstrates how you can run inference on Gemma 3 model using Node.js and [Transformers.js](https://huggingface.co/docs/transformers.js/index). Transformers.js lets you run Hugging Face's transformer models directly in browser, offering a JavaScript API similar to Python's.  It supports NLP, computer vision, audio, and multimodal tasks using ONNX Runtime and allows easy conversion of PyTorch, TensorFlow, and JAX models.\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/Gemma/[Gemma_3]Using_with_Transformersjs.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ftkrrn3aZyAl"
      },
      "source": [
        "## Setup\n",
        "\n",
        "### Select the Colab runtime\n",
        "To complete this tutorial, you'll need to have a Colab runtime with sufficient resources to run the Gemma 3 model. In this case, you can use CPU runtime:\n",
        "\n",
        "1. In the upper-right of the Colab window, select **▾ (Additional connection options)**.\n",
        "2. Select **Change runtime type**.\n",
        "3. Under **Hardware accelerator**, select **CPU**."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eCJ7yo3-Zzdj"
      },
      "source": [
        "## Installation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ET_KH77YZ5lc"
      },
      "source": [
        "Let's get started with installing the dependencies."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "d7Ds4h2q0ItU"
      },
      "outputs": [],
      "source": [
        "# Install Node.js\n",
        "!curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -\n",
        "!sudo apt-get install -y nodejs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ObI8d_Rwa_nn"
      },
      "source": [
        "## Create Node.js project\n",
        "\n",
        "Create a new Node.js project and install the required transformers package via [NPM](https://www.npmjs.com/package/@huggingface/transformers)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "5nmPcg5J0cYj"
      },
      "outputs": [],
      "source": [
        "# Create project directory\n",
        "!mkdir gemma3-node\n",
        "%cd gemma3-node\n",
        "\n",
        "# Initialize NPM project\n",
        "!npm init -y\n",
        "!npm i @huggingface/transformers"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "gGYVoOXA3T7-"
      },
      "outputs": [],
      "source": [
        "%%writefile package.json\n",
        "\n",
        "{\n",
        "  \"name\": \"gemma3-node\",\n",
        "  \"version\": \"1.0.0\",\n",
        "  \"main\": \"index.js\",\n",
        "  \"type\": \"module\",\n",
        "  \"scripts\": {\n",
        "    \"test\": \"echo \\\"Error: no test specified\\\" && exit 1\"\n",
        "  },\n",
        "  \"keywords\": [],\n",
        "  \"author\": \"\",\n",
        "  \"license\": \"ISC\",\n",
        "  \"description\": \"\",\n",
        "  \"dependencies\": {\n",
        "    \"@huggingface/transformers\": \"^3.4.1\"\n",
        "  }\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U9mJ6Pi3bxCY"
      },
      "source": [
        "## Transformers.js Inference\n",
        "\n",
        "Now, let's run inference on the Gemma 3 model using Transformers.js. First, create a text generation pipeline and then prepare input (list of text messages) to run inference and get the output as desired response. For reference, you can check the model's page on the Hugging Face model hub under ONNX models section [here](https://huggingface.co/onnx-community/gemma-3-1b-it-ONNX)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "-Gy2LdaY3Iqh"
      },
      "outputs": [],
      "source": [
        "%%writefile index.js\n",
        "\n",
        "// Import the required modules\n",
        "import { pipeline } from \"@huggingface/transformers\";\n",
        "\n",
        "// Create a text generation pipeline\n",
        "const generator = await pipeline(\n",
        "  \"text-generation\",\n",
        "  \"onnx-community/gemma-3-1b-it-ONNX\",\n",
        "  { dtype: \"q4\" },\n",
        ");\n",
        "\n",
        "// Define the list of messages\n",
        "const messages = [\n",
        "  { role: \"system\", content: \"You are a helpful assistant.\" },\n",
        "  { role: \"user\", content: \"Write me a short poem about Machine Learning.\" },\n",
        "];\n",
        "\n",
        "try {\n",
        "  // Generate a response\n",
        "  const output = await generator(messages, {\n",
        "    max_new_tokens: 128,\n",
        "    do_sample: false,\n",
        "  });\n",
        "  console.log(output[0].generated_text.at(-1).content);\n",
        "} catch (error) {\n",
        "  console.error(\"Error generating response:\", error);\n",
        "}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zz81XOKebf_j"
      },
      "source": [
        "## Run Application"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "id": "PmVUVFDf3-yE"
      },
      "outputs": [],
      "source": [
        "# Run the node.js application\n",
        "!node index.js"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gFgOMvLmcVnY"
      },
      "source": [
        "## Conclusion\n",
        "\n",
        "Congratulations! You have successfully run inference on Gemma 3 model using Transformers.js via Node.js environment. You can now integrate this into your projects."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "[Gemma_3]Using_with_Transformersjs.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}